Joint architecture and knowledge distillation in CNN for Chinese text recognition

The distillation technique helps transform cumbersome neural networks into compact so that models can be deployed on alternative hardware devices. main advantage of distillation-based approaches include a simple training process, supported by most off-the-shelf deep learning software and no special requirements. In this paper, we propose guideline for distilling the architecture knowledge pretrained standard CNNs. proposed algorithm is first verified large-scale task: offline handwritten Chinese text recognition (HCTR). Compared with CNN in state-of-the-art system, reconstructed reduce computational cost >10×and model size >8×with negligible accuracy loss. Then, conducting experiments two additional classification task datasets: Text Wild (CTW) MNIST, demonstrate approach also successfully applied mainstream backbone networks.